Engagement Survey Technical Report
Eagle I.O
Most recently updated 2021-07-20
Home
Placeholder
Introduction
Definitions of Engagement
The roots of employee (aka work; e.g., W. Schaufeli & Bakker, 2010) engagement research likely started with theoretical expansions of forms of employee participation (see, for example, Ferris & Hellier, 1984) and job involvement (e.g., Elloy et al., 1991). This exploration extended into broader considerations of attitudes and emotions (Staw et al., 1994) and were informed by further exploration of the dimensionality of constructs such as organizational commitment (Meyer & Allen, 1991). The 1990’s saw focused development and refinement (for example, a dissertation; Leone (1995) or actual semantic reference; Kahn (1990)). Staw et al. (1994) investigated the relationships between positive emotions and favourable work outcomes, and although they do not use the word, “engagement,” their distinction between felt and expressed emotion was likely a stimulating influence upon the burgeoning interest in the engagement construct. Gallup is also widely acknowledged as an early pioneer in the measurement of the construct (see, for example, Coffman & Harter, 1999).
Kahn (1990) described engaged employees as being physically involved, cognitively vigilant, and emotionally connected. Although occasionally referred to as residing on the opposing pole to burnout (Christina Maslach & Leiter, 2008), these two constructs are currently most commonly conceptualized as being distinct (Timms et al., 2012), although not universally (Cole et al., 2012; Taris et al., 2017).
Engagement/Burnout: Burnout is defined as a psychological syndrome characterized by exhaustion, cynicism, and inefficacy, which is experienced in response to chronic job stressors. Engagement is understood to be the direct opposite of burnout and exist on a continuum—with engagement on one end and burnout on the other. Exhaustion (low energy), cynicism (low involvement), and inefficacy (low efficacy) are characteristic of burnout; whereas, high energy, high involvement, and high efficacy are characteristic of engagement C. Maslach & Leiter (1997) (Maslach and Leiter, 1997) (Leiter & Maslach, 2004).
Work engagement: Work engagement refers to a positive, fulfilling, work-related state of mind that is characterized by vigor, dedication, and absorption. Vigor is characterized by high levels of energy and mental resilience while working. Dedication refers to being strongly involved in one’s work and experiencing a sense of significance, enthusiasm, inspiration, pride, and challenge. Absorption is characterized by being fully concentrated and happily engrossed in one’s work, whereby time passes quickly and one has difficulties with detaching oneself from work (Wilmar B. Schaufeli et al., 2002) - The Utrecht Work Engagement Scale (UWES), a self-report questionnaire first developed by Schaufeli and Bakker (2003) (W. B. Schaufeli & Bakker, 2003) includes the three aspects of work engagement: vigor, dedication, and absorption. - we have to make note of how we are different from this!
Employee engagement: Employee engagement refers to the ‘‘individual’s involvement and satisfaction as well as enthusiasm for work’’ (James K. Harter et al., 2002) Work engagement is the mental state where employees…
…feel full with physical energy (Vigor)
…are enthusiastic about the content of their work and the things they do (Dedication)
…are so immersed in their work activities that time seems to fly (Absorption)
The tripartite substantive model of employee engagement is also partially informed by the definitions provided with the Utrecht Work Engagement Scale (Wilmar B. Schaufeli & Bakker, 2003).
Instrument creation
Placeholder
Intended Structure
Item generation
Content Validation
Procedure
Placeholder
Pilot
Sampling strategy (pilot)
Pilot analyses
Demographic Information
Analyses
Pilot analyses
282 people responded. We may want to further screen these 282 responses for short response durations. According to communications with Qualtrics in early December, this is the number of seconds to complete the entire survey. Currently, the “number of people who clicked on the assessment link (330)” versus valid n (282) may take care of our very low duration respondents. The shortest response duration in the 330 datafile is 101 whereas the lowest in the 282 datafile is 43. All analyses were performed via R version R version 4.1.0 (2021-05-18) (R Core Team, 2021).
Classical test theory
In addition to the below interactive plot [via plotly version 4.9.4.1; Sievert et al. (2021)], a full inter-item correlation matrix is located in Appendix @ref(corrs).
## Some items ( I find it difficult to mentally disconnect from work. ) were negatively correlated with the total scale and
## probably should be reversed.
## To do this, run the function again with the 'check.keys=TRUE' optionInternal consistency estimates
Condition 1 administered items within the substantive dimensions (with successive randomized blocks of Cognitive, Affective, and Behavioral items). Condition 2 administered items within the attitudinal dimensions (with successive randomized blocks of Absorption, Vigor, and Dedication items). Condition 3 stressed the substantive dimensions (with items fully randomized regardless of attitudinal association). Condition 4 stressed the attitudinal dimensions (with items fully randomized within attitudinal dimension regardless of substantive scale association, see Chapter @ref(conds) and Appendix @ref(pilot2)). All internal consistency estimates were generated via psych version 2.1.6 (Revelle, 2021). Alphas for the candidate 12-item scales were:
| Dimension | Undifferentiated | Condition 1 | Condition 2 | Condition 3 | Condition 4 |
|---|---|---|---|---|---|
| Affective | 0.87 | 0.88 | 0.84 | 0.88 | 0.87 |
| Behavioral | 0.79 | 0.73 | 0.84 | 0.76 | 0.87 |
| Cognitive | 0.81 | 0.79 | 0.85 | 0.8 | 0.87 |
| Absorption | 0.74 | 0.72 | 0.77 | 0.72 | 0.87 |
| Vigor | 0.85 | 0.8 | 0.88 | 0.84 | 0.87 |
| Dedication | 0.9 | 0.9 | 0.91 | 0.91 | 0.87 |
“Cell” level alphas (4 items each scale, responses collapsed across administrative conditions) were:
| Cell | Alpha |
|---|---|
| Affective - Absorption | 0.66 |
| Affective - Vigor | 0.71 |
| Affective - Dedication | 0.75 |
| Behavioral - Absorption | 0.56 |
| Behavioral - Vigor | 0.7 |
| Behavioral - Dedication | 0.64 |
| Cognitive - Absorption | 0.59 |
| Cognitive - Vigor | 0.62 |
| Cognitive - Dedication | 0.83 |
Corrected item-total correlations are presented in Appendix @ref(rdrops)
Confirmatory factor analyses
We used lavaan version 0.6.9 (Rosseel et al., 2021) and semPlot version 1.1.2 (Epskamp, 2019)
| Model | \(\chi^2\) | df | RMSEA | SRMR | CFI | TLI | AIC |
|---|---|---|---|---|---|---|---|
| 3-factor substantive | 2159.21 | 591 | 0.11 | 0.1 | 0.64 | 0.62 | 25481.97 |
| Hierarchical substantive | 2159.21 | 591 | 0.11 | 0.1 | 0.64 | 0.62 | 25481.97 |
| 3-factor attitudinal | 2318.92 | 591 | 0.11 | 0.1 | 0.6 | 0.58 | 25641.68 |
| Hierarchical attitudinal | 2318.92 | 591 | 0.11 | 0.1 | 0.6 | 0.58 | 25641.68 |
Multigroup analyses
- using experimental condition as “multiple groups” (measurement invariance)
Bifactor analyses
bifactor analysis are most commonly applied in the exploration of common method variance (see, for example, Reise, 2012; Rodriguez et al., 2016). Giordano et al. (2020) provide an overview regarding past and potential applications of exploratory bifactor analysis and cite Reise (2012) as an influential impetus for the resurgence of bifactor models in general.
## lavaan 0.6-9 did NOT end normally after 820 iterations
## ** WARNING ** Estimates below are most likely unreliable
##
## Estimator ML
## Optimization method NLMINB
## Number of model parameters 114
##
## Used Total
## Number of observations 239 282
##
## Model Test User Model:
##
## Test statistic NA
## Degrees of freedom NA
##
## Parameter Estimates:
##
## Standard errors Standard
## Information Expected
## Information saturated (h1) model Structured
##
## Latent Variables:
## Estimate Std.Err z-value P(>|z|)
## Cognitive =~
## Item_1 1.000
## Item_2 0.516 NA
## Item_3 1.090 NA
## Item_4 0.353 NA
## Item_13 1.130 NA
## Item_14 0.838 NA
## Item_15 0.798 NA
## Item_16 1.098 NA
## Item_25 0.988 NA
## Item_26 0.909 NA
## Item_27 1.080 NA
## Item_28 0.999 NA
## Affective =~
## Item_5 1.000
## Item_6 0.656 NA
## Item_7 0.475 NA
## Item_8 1.242 NA
## Item_17 0.933 NA
## Item_18 1.372 NA
## Item_19 1.044 NA
## Item_20 0.536 NA
## Item_29 0.841 NA
## Item_30 0.381 NA
## Item_31 0.724 NA
## Item_32 1.063 NA
## Behavioral =~
## Item_9 1.000
## Item_10 1.109 NA
## Item_11 0.572 NA
## Item_12 1.176 NA
## Item_21 1.362 NA
## Item_22 1.573 NA
## Item_23 0.873 NA
## Item_24 0.890 NA
## Item_33 0.707 NA
## Item_34 0.879 NA
## Item_35 1.237 NA
## Item_36 0.369 NA
## Absorption =~
## Item_1 1.000
## Item_2 19131.383 NA
## Item_3 -390.734 NA
## Item_4 21074.473 NA
## Item_5 2937.496 NA
## Item_6 -1569.896 NA
## Item_7 727.592 NA
## Item_8 -1533.787 NA
## Item_9 4326.028 NA
## Item_10 5122.173 NA
## Item_11 -1534.018 NA
## Item_12 -1356.265 NA
## Vigor =~
## Item_13 1.000
## Item_14 -3.413 NA
## Item_15 0.449 NA
## Item_16 -0.517 NA
## Item_17 -0.246 NA
## Item_18 -1.152 NA
## Item_19 0.766 NA
## Item_20 -3.510 NA
## Item_21 0.381 NA
## Item_22 -0.278 NA
## Item_23 0.324 NA
## Item_24 0.093 NA
## Dedication =~
## Item_25 1.000
## Item_26 1.378 NA
## Item_27 1.009 NA
## Item_28 0.830 NA
## Item_29 0.885 NA
## Item_30 0.650 NA
## Item_31 0.274 NA
## Item_32 0.512 NA
## Item_33 0.010 NA
## Item_34 0.122 NA
## Item_35 0.888 NA
## Item_36 0.783 NA
##
## Covariances:
## Estimate Std.Err z-value P(>|z|)
## Cognitive ~~
## Affective 0.549 NA
## Behavioral 0.363 NA
## Affective ~~
## Behavioral 0.372 NA
## Absorption ~~
## Vigor 0.000 NA
## Dedication -0.000 NA
## Vigor ~~
## Dedication -0.082 NA
## Cognitive ~~
## Absorption 0.000
## Vigor 0.000
## Dedication 0.000
## Affective ~~
## Absorption 0.000
## Vigor 0.000
## Dedication 0.000
## Behavioral ~~
## Absorption 0.000
## Vigor 0.000
## Dedication 0.000
##
## Variances:
## Estimate Std.Err z-value P(>|z|)
## .Item_1 1.343 NA
## .Item_2 0.350 NA
## .Item_3 0.786 NA
## .Item_4 0.124 NA
## .Item_13 0.809 NA
## .Item_14 0.659 NA
## .Item_15 1.471 NA
## .Item_16 0.760 NA
## .Item_25 1.350 NA
## .Item_26 0.629 NA
## .Item_27 1.282 NA
## .Item_28 0.763 NA
## .Item_5 1.324 NA
## .Item_6 1.195 NA
## .Item_7 0.845 NA
## .Item_8 0.688 NA
## .Item_17 0.484 NA
## .Item_18 0.559 NA
## .Item_19 0.771 NA
## .Item_20 0.780 NA
## .Item_29 0.479 NA
## .Item_30 1.514 NA
## .Item_31 0.538 NA
## .Item_32 0.667 NA
## .Item_9 1.293 NA
## .Item_10 1.961 NA
## .Item_11 1.288 NA
## .Item_12 1.342 NA
## .Item_21 0.894 NA
## .Item_22 0.510 NA
## .Item_23 0.338 NA
## .Item_24 1.026 NA
## .Item_33 0.401 NA
## .Item_34 0.532 NA
## .Item_35 0.547 NA
## .Item_36 0.984 NA
## Cognitive 0.593 NA
## Affective 0.639 NA
## Behavioral 0.283 NA
## Absorption 0.000 NA
## Vigor 0.072 NA
## Dedication 0.596 NACurrently pretty messed up prior to item deletion (Vigor especially). Model also technically not converging need Renata to fix.
Deese guys also do bifactor stuff: Mansolf & Reise (2017)
Summary
Recommendation for final instrument based on consideration of all of the above pieces of evidence
Future plans
Placeholder
Things to do
Research projects
Author bios
|
Casey Osorio-Duffoo received his M.A. in Industrial and Organizational Psychology from Montclair State University. He enjoys stealing chicken eggs from hen coops and outwitting crows. [NOTE. Currently being held against own will in avian rehab] |
| Renata Garcia Prieto Palacios Roji is pursuing her Ph.D. in Industrial and Organizational Psychology at Montclair State University. She enjoys allfallfa, the smellll of llavender, and glleefulllly prancing through dew-filllled meadows. |
|
|
John Kulas received his Ph.D. in Industrial and Organizational Psychology from Northern Illinois University. He feels ashamed of his previous obsession with the evisceration of the mancub Mowgli, and is currently easing into a vegan lifestyle, slowly weaning himself from a diet consisting primarily of foxes and llamas. |
References
Placeholder